Repository navigation
fix: preserve unread input in local streaming writes - #295
Open
perfloop[bot] wants to merge 3 commits into
Open
perfloop[bot] wants to merge 3 commits into
perfloop[bot] wants to merge 3 commits into
Conversation
Signed-off-by: Perfloop Agent <agent@perfloop.ai>
Signed-off-by: Perfloop Agent <agent@perfloop.ai>
Signed-off-by: Perfloop Agent <agent@perfloop.ai>
[Medium impact] This PR appears safe to merge. SummaryLocal streaming writes preserve the reader’s starting position.
Reviews (1) · Last reviewed commit: "fix: check cancellation before reader po..." · Reviewed by Greptile |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Local PutStream writes should consume only the bytes a caller has not already read. The local first attempt starts at the reader's call-entry position, and retries return to that same position. A positioned reader that relied on the old local rewind will now store only its unread suffix. For active calls whose starting position cannot be obtained, storage gets one attempt; a failed retry seek returns an error instead of writing a different slice. No public signature or wire format changes, and remote forwarding is unchanged. The embedded API comment and guide document the local behavior.
TestPutStreamReaderPositioncovers positioned bytes.Reader and file inputs, zero and end positions, reader-only and pipe-backed readers, cancellation, and a Seeker whose position query blocks.TestPutLocalReaderRetryPositioninjects a retryable failure after a partial read and checks replay.TestPutLocalReaderWithoutPositionAttemptsOnceandTestPutLocalReaderRetrySeekFailurecover unavailable positions and failed restoration. The focused server Put suite exercises the unchanged unary bytes-reader and io.PipeReader callers.This condition holds with this change and does not hold at
0759cbea25127e1e69a2ac0bb5a25bc429d32ab8:With a live context and a local embedded.Client.PutStream call, a successful write stores exactly the bytes from the reader's call-entry position through EOF; retryable storage failures replay from that offset when it is available, while readers with no obtainable starting position receive one storage attempt from the current position. A failed retry seek returns an error without committing another slice, and an already-canceled seekable call returns a timeout before invoking the reader's Seek method, without consuming or storing the input..A standalone executable also drives embedded.Client.PutStream through a retryable metadata-commit failure after the body is consumed. The executable and its fault-injection helper are not included in the PR. The shipped operations test covers the separate partial-read retry case. The focused embedded and server Put tests passed, including the race-filtered Put path.
TestPut_MetadataCommitFailureReclaimsRawFilepassed; it verifies that a raw file is reclaimed after a failed metadata commit. The server build, lint, and license check also passed.Caller incidence and storage or copying cost remain unmeasured.
Generated by Perfloop.
Note
Medium Risk
Changes core local ingest semantics for partially consumed readers and retry paths; behavior shift for callers that previously relied on full-stream rewind on retry.
Overview
Local streaming puts no longer rewind seekable readers to offset zero.
PutStream/PutLocalnow store bytes from the reader’s call-entry position through EOF. The first write reads forward from there; on retryable storage failures,putLocalWithRetryrestores that captured offset viaio.Seekerinstead of seeking to the start.Readers without a reliable position get one storage attempt. A failed retry reposition returns an error (and is not treated as retryable storage). Canceled contexts return a timeout before querying position on seekable inputs, without consuming the body or creating a key.
Docs and
Client.PutStreamcomments describe this local behavior. New embedded integration tests andput_local_retry_testcover positioned inputs, pipes, cancellation, and retry/seek edge cases. Remote gRPC forwarding is unchanged.Reviewed by Cursor Bugbot for commit c9fed33. Bugbot is set up for automated code reviews on this repo. Configure here.